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PREFACE 



Work on this thesis was begun during the first eleven weeks of 195U 
while the writer was employed by the electronics design and oanufac tun- 
ing fina of Gilfillan Brothers, Incorporated, located at Los Angeles, 
California. This employment was the industrial experience tour in the 
Engineering Electronics Curriculun of the United States llaval Postgrad- 
uate School which requires the student to work as a junior engineer in 
the electronics company of liis choice. Completion of the thesis was 
accomplished during the Spring of 1?5U at the United States Naval Post- 
graduate School. 

This investigation was done in cooperation with Hr. Louis A. Ule, 
a member of the Theoretical Analysis Group in the Engineering Department 
of Gilfillan Brothers, Incorporated. Its purpose was to round out the 
least squares smoothing and prediction theory embodied in Mr. Ule's 
in tr a- company neirorandun of January 12, entitled, "Polynomial 

Filters. " While a considerable amount of material in the thesis is the 
result of joint discussion or discovery, the formulation of the weighting 
function (Chapter XI, Section U) and the proof of the steady state filter 
response in the absence of noise (Chapter II, Section 5 ) are abstracts 
of material written by Hr. Ule. 

The writer wishes to express his appreciation to Hr. Ule for his 
guidance, assistance, and original material. He is also indebted to 
Professors Abraham Sheingold and Robert Kahal of the United States Naval 
Postgraduate School Staff for their suggestions anu encouragement while 
acting as thesis advisers and readers. 
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SY1I0PSIS 



The smoothing anci prediction problem is one of orctracting fron a 
noisy signal the best estimate of some functional of the true signal at 
some past present, or future time. The historical development of the 
least squares solution to the problem is briefly discussed along with 
other background information relating to this process. 

A method of smoothing and prediction is developed which is a dynamic 
form of least squares curve fitting. The true signal is assumed to' be 
composed of completely foreknown functions of time each multiplied by a 
constant coefficient. The filter weighting function is chosen so that 
the filter output will be composed of the same functions as the true 
signal except for a possible translation in time, differentiation, or 
integration. The squared error is time weighted before being minimised. 
Ymile tlie derivation of the weighting function is independent of the type 
of noise, the choice of the squared error weighting function may depend 
upon the character of the noise. Functions which car be treated are found 
to be the solutions of the reduced equation of a linear differential 
equation vrith constant coefficients. The resultant filter weighting 
function depends upon the true signal, the prediction time, the squared 
error weighting, and an inverse matrix w! ich is the solution of a set of 
integral equations. 

Tlie resultant filters arc constructed of linear elements which do not 
vary with tine. Although no conditions for physical realizability are 
imposed, a high probability exists that the filters may be constructed by 
one of several methods. A step by step procedure for the filter design is 
given together vrith specific ercanples of its use. A phenomenon akin to 
resonance is observed for exponential and polynomial functions. 
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CHAPTER I EITRODUCTIOil To THE SHOOTiHIIG AND FREDICTIUL PROBLEM 



1. Description of the Problen. 

The smoothing problem is one of extracting true signals from noisy 

p 

ones. Thi3 problen lias be on described by Bode and Shannon and by Zadoh 
and Ragassini^O. it is usually asouned that a given perturbed signal, 
R(t), is the cun of a true signal, P(t), and a random disturbance or 
noise, H(t). 



R(t) = P(t) + N(t) 



( 1 ) 



By performing appropriate operations on R(t), it is desired to obtain a 
result Q(t) v;hich is the best approximation of P(t). Tins smoothing 
pi'oeess nay be cor.bincd uith prediction to provide a continuous approxs- 
ination of P(t) at some past or future tine. In other v/ords, given R(t) 
and a fixxcd preaiction tine 8, vdiat is P(t f 8)? 8 nay be positive, 

negative, or zero depending upon vjhethcr prediction, tine delay, or 
smoothing alone is desired. Both smoothing and prediction nay be fur- 
ther combined vrith differentiation and integration so that in the ger>- 
eral case some functional of P(t + 8) is obtained. Commonly desirod 
functionals are of the form P(t + 9 ), dp(t + 9)/dt and P(t f 8) dt. 
The problen of filling the box: marked n ?" is illustrated in Figure !• 



R(t) 



P(t) + H(t) 



“Optimum** 

Transfer 

Operator 

? 



vj(t) = best approximation 
of functional of P(t f 8) 



Figure 1. The Smoothing and Prediction Problem. 
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As a class, tne plysical counterparts of the "optimum" transfer 
operators ai'c commonly known as s. oct. ling and i redacting filters, c sti- 
lus tors, oi' predictors. Since ary one filter i .ay not perform all of the 
aforesaid processes, specific names to indicate cenctly wliat cue.' filter 
docs light be in order. ihianplcs of ouch ns-.ies would be "reproducer" to 
indicate a a-iootlirg process v.ith no tine lac or lead; "differentiatinc 

1. redictor" to indicate both differentiation and snootiiing with a fined 

V 

tii:.e lead; and "integrating delayer" to indicate both intec ration and 
sicoothing 'with a fined tine delap-. 

The desicn of an optimum predictor involves two questions. In the 
first place, wliat are the transfer charactci*iotico for the "optimal" 
ti'aiisfor operator? Secondly, how nap/ tnese characteristics be physically 
realized? Before the first question can be answered, a criterion of 
performance must be chosen and a method of characterizing the sicnal and 
noise mist be found. By choosing a realistic criterion, the answer to 
botli questions nry be greatly simplified. Tlie proper characterization 
of the signal and noise will per.lt one to mhe the optinun use of the 
a priori information conccminc the signal and noise. 

uldie tiie smootliing and prediction _ roolcn is usually thought of as 
one in communication theory, it light well be generalized to cover such 
diverse fields as gunnery, economic prediction, weather forecasting and 
statistics. 

2. Performance Criteria. 

To attach the problem mathematically, a criterion of performance or 
measure of fidelity ..list be chosen for the optimum predictor. Tils 
criterion should be realistic i order tliat the results liave meaning. 
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It should also be si.rple in order tiiat tlic nathc. at leal treatment nay be 
economically possible. normally negative errors are as undesirable as 
positive or.es so that the chosen criterion ohould be an even function of 
error. 



transient response in linear transfer systems called "duplicators, " in 
v/hich tire outixit approrinateiy duplicates tire input. They have concluded 
that tire minimum intccral of tire time-multiplied absolute value of error 
criterion is best. 



The results of this article do not appear to meet the requirements of 
the cnoo tiring and prediction problem since the lack of a noise component 
in the input signal has been tacitly assumed. 

For signals which consist of a numoer of discrete pieces of data, 
the jjrobability that an error erdsts might be used as a measure of 
fidelity. It is meaningless in the case of continuous signals in the 
presence of noise since the probability that an error vri.ll occur at any 
given point is air, ays one. 

If tire error is designated as e(t) and the mean value of a quantity 
ty a bar above tlrat quantity, a performance criterion for continuous 
signals might employ the minimization of one of the following: 




I 




( 2 ) 





where n is a positive integer 



probability that |e(t)| a specified limit 
probability that mission is not accomplished 




found wiuc usage. Tils criterion requires tliat: 



0 2(t) = — i 



T, 



2 




1 



( 3 ) 



From equation (3) it is evident that the undesirability of an error is 
weighted according to the square of its magnitude. Tiie principle altera- 
tion is paid to looking the very large errors as snail as possible "with 
little record to snail errors. V/itliin the tine rance of interest the 
weighting is independent of the tine when the error occurs. 

There are several reasons ray this criterion lias found such vdde 
acceptance. First of all, it is cenerally applicable and in reasonable 
accord with physical requirenents. Ilunerous cases e:d.st in which the 
undesirability of an error increases T/ith its ixignitudc. The neon-square 
error criterion is reasonable when the error lias a normal or Gaussian 
distribution which is independent of tine, a condition ofter encountered 
in practice. In the second place, the nean- square error netliod elird- 
nates nuch of the nathenatical complexity involved in the use of other 
criteria. A liighly developed body of nathenatical Imowlodge lias been 
built around the idea of the nean-square value. In sone cases, it is the 
only criterion vrldch is anenablc to nathenatical treatment. Finally, 
tiiis criterion leads to useful results. Information can often be ob- 
tained fron tiiis method even when it is not strictly applicable. 

The nean-square criterion does not always meet the requirements of 

25 

the systen. Ragazzini and Zadeli have presented the case in which all 



5 



errors larger ti;an a certain Unit arc oguaiiy sad, wldlc tnose less 
tlian the linit are equally acceptable. Tliis is the classic gunnery 
problem best described by the a;dx>n, "A miss is as good as a mile. '• 

Cases nay occur in which the undosirability of an error depends upon 
the tine at wniclr the error occurs. For c?:amp lc, the undesirability 
of an error during a transition fron one .roue of operation to another 
nay be quite different fron that while operating in a given node. 

Another case in which the nc an- square error criterion would not apply 
involves the tines when as nary very accurate predictions as possible 
are desired and an occasional gross error is of little consequence. 

3* Historical Development. 

The first successful attempt in this country to rut the noise 

29 

problem on a mathematical basis was Norbert Wiener's^ classic report 
for the National Defense Research Council entitled, "The Ertrapolation, 
Interpolation, and Smoothing of Stationary Time Series, " which appeared 
in February'- 19U2. An earlier, similar but not identical article by 
A. I!. Kolmogoroff^ appeared in a Russian scientific journal in 19hl* 
Although both worl:s are devoted to a study of the optimum prediction 
problem through the use of statistical methods and support each other, 
they are tire result of independent investigations. 

Tire IViencr-Kolnogoroff theory is based on tlrrcc assumptions which 
limit its range of application. 

1. Tire signal and noise m ay be satisfactorily represented by stationary 
time series, that is, tire statistical properties of the signal and noise 
do not change with time. 
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2. Tile i xin i rmi mean- square error is a reasonable criterion of perform- 
ance. The ensemble average is taken over all \ os si ole signal and noise 
functions, oach weighted according to its probability of occurance. 

3. The operation used for smoothing and prediction is linear so that 
it nay be performed by a physically realizable filter. 

For these reasons the theory has been described as "linear’ least square 
prediction and smootiring of stationary tine series." If a given tine 
scries is stationary and the correlation functions of the signal and 
noise are lurovm, this theory nay be used to obtain the specifications 
for an optimum predictor. 

An early application of Wiener 1 s work ty Pliillips and Weiss 
eirtended the theory to quasi- stationary processes, that is, processes 
which arc stationary in the range of interest. The problem under 
investigation was the smootiring of data obtained in the tracking of a 
gunnery target. In this case, the signal and noise characteristics 
are considered invariant except when the target changes course. Then 
the error is large enough so that the possibility of a hit is negli- 
gible and the results need not be considered. 

Because of the formidable mathematics involved in Wiener’s theory, 
21 22 

Homan Levinson 3 Iras written two papers with a view toward facili- 
tating the computational procedure and summarizing the significant 

2 

results. Bode and Shannon have presented a simplified derivation of 
the theory which does not resort to the use of integral equations or 

autc— correlation functions. Recent books on information theory, such 

7 1 

as those by Goldman and Bell , discuss tire noise problem. 

30 

Zadeh and Ragazzini have further e:rtended tire theory in two ways. 
The desired signal nay also contain a non- random function of time which 



7 



is reproscrrtable as a polynomial of decree not creator than n and about 

■which no infoi’ration other than n is lenovm. In addition, the in; ulsivc 

response of the predictor nay vanish outside a spocificd tine interval 

0 t ^ T, whereas T is infinite in Y/icner's theory, 

12 

A recent paper by Hauser discusses the geometric aspects of the 
least squares smoothing process. 

U. Tine versus Frequency Donain Linoar Filters. 

The most important prerequisite of ary filter is tlat it be plysi- 
cally realisable. Unless this condition is net, any “optimum" transfer 
operation is of academic interest only. For case of mathematical 
treatnent and practical realizability, it is usually desirable that the 
filter also be fixed and linear. Janes and Weiss‘S have indicated the 
characteristics of the normal response of a linear filter. The normal 
response should bo a linear function of the input so tliat the linoar 
superposition of inputs and resultant responses vdll hold. It should 
depend only on the past and present values of the input. In addition, 
this response should be independent of the tine origin. This noans 
that the values of circuit elements are fixed and independent of tine. 

The classical approach to the problem of separating signals fron 
interfcrcr.ee, whether due to noise or other signals, is to provide a 
filter of limited bandwidth as determined by the steady state frequency 
components of the desired signal. The resonant tuned circuit for sinu- 
soidal signals is a good example of this technique. For signals of a 
transient nature which have neither bandwidth nor frequency continents 
in the usual sense, this method nay be clumsy. 

With the increased use of the operational mathematics, it lias become 
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conventional to utilise the Laplace transform vi.cn dealing -,/ith filters 
for transient signals. The situation is represented by Figure 2. 



F(a) 



■*“ H(s) 



G(s) 



Figure 2. Frequency Domain Filtering. 



F(s) and G(s) are the Laplace transfoms of the input and output 
signals respectively. Hie transfer function of the filter H(s) is 
defined as the ratio of the Laplace transforms of aiy normal response 
and the input tliat produces it. It follows tiiat 



The filtering of signals vliich are functions of time can also be 



filter is completely described by its weighting function YJ(t). Tliis 
weighting function, often called tiie impulsive response of tne filter, 
is the normal response of the filter to a unit a_.pulse applied at zero 
tine. The weighting function egresses the cirtcnt to widen the distant 
past of the input affects the present filter res. once and hence nay be 
thought of as the memory of the filter. The output of the filter, g(t), 
is given by the convolution or superposition integral in ter... s of x(t), 
W(t), and a dum.y valuable of integration V , 



G(s) = F(s) H(s) 



(U) 



done directly' in the tine domain. The input signal is f(t) and tlie 




^ 0 

Tine domain fLltcidrg is illustrated in Figure 3« 
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*- c(t) 



f(t) 



W(t) 



Figure 3- Tine Domain Filtering • 

Hie follmving co rre spondenc e ejdlsts between the tine and frequency 
domain filters* 



Tliis duality between tlie tine and frequency donains is further shown by 



A teclmique utilising both the tine and fre x uency donains wall be 
used later. It consists of finding the appropriate weighting function, 
Yf(t), and then talcing its Laplace transform to obtain 11(c) • In this 
case the properties of the filter are more readily found in the time 
domain. The switch to the frequency domain is employed so that con- 
ventional methods of network synthesis my be used. 

5. Numerical Least Squares Curve Fitting. 

Related to the discussion of the second section of this chapter i3 

26 

the static case of least squares curve fitting. Scarborough lias dis- 
cussed both tlie basis for the method and the steps necessary to cany 
out the numerical computation. It is based on the Normal Law of Error, 




( 6 ) 




(7) 




^(t) * f 2 (t) « F l (s) F 2 (s) 
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the Principle oi' Least Squares, and the Lav/ of Error for Residuals. 
Consider the function, P, vs-icli is the sun of several functions, f^(x), 
... f n (x). 

P = • b x f^x) + b 2 f 2 (x) + ... + b n f n (x) (8) 

Pairs of observed values, (x, , P ), (x 0 , P„), • . . (x , P ), aro ot>- 
tainea Iron a set of m raasurcixnts of equal precision. It is desired 
to find the values of the coefficients, b^, b . . . b^, such that a 
Graph of P trill corse as near as possiolc to each of the n points. A 
set of li residual equations is obtained by substituting each of tire 
pairs of values in turn into equation (8) . 

V 1 = + b^f 2 (:s L ) + . . . f b n £ n (»j_) “Pi ( 9 ) 

v 2 = ^l^) f b L f 2^ f * * * + “ p 2 (10) 



v n = Vl<*a> + b 2 W + • • * + b nW " P n <*> 

Hie best values of the unlcnov.n coefficients are tuose iviiich mice 
tlie cun of the squares of the residuals a nininun. 

f v 2 2 + . . . + v ^ =■ f(b 1 , b 2 , . . . b n ) = nininun (12) 

To nal:c equation (12) a naidxiuu or nininun, the partial derivatives of 

f(b^, b^, . . . b ) with respect to each oi tlie coefficients snail be 
12 n 

zero. Tlie values of the residuals, v , v_, . . . v , given in equations 

X L- d 

( 9 ) tlirough (11) are substituted into equation (12) j the partial differ- 
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entiations are pcrforraod with res ect to b., b rJ . . . b ; arjd the n 

1 n 

resultinc equation a aro equated to aero. After divicLnic tlirouch by 2, 
a set of siraultancous equations, called non.nl equations, are obtained. 
They my be solved by ordinary algebra. 



W[ b iW + + • • • + Vn ( i> - p x] 



+ f 1 ( x 2 ) 

♦ w 



b f (o: ; ) 4- b f (x ) f . . . f b f (x ) - P 

lid 4.dd n n 2 2 



b £ ' (x ) + b £ (x ) f . . . f b f (x ) - P ' 
lira 22n nnn n 



f . . . 

= 0 



(13) 



: (5 i)[ b ii (3 



O + b 0 f (x ) + . . . 



+ f 0 (x„) 
+ f 9 (x ) 

d U 



+ b f (x,) - P,~l 
n n l ±J 



b f (x 0 ) + b f n (sj + . . . f b f (x ) - p 

± j. l. u c. l. n n c. c 

b f (x ) + b f (x ) * . . . + b f (x ) - P 

lln d d n nnm n 



+ • • • 

= 0 



(111) 



f n (: i ) [Vl ( i ) + V 2 <*1> + • ' ’ + b »W - P l] 



+ W 

♦ £ n ( i> 



b l f l (:C 2^ + b 2 1 2^ X 2^ + 
b f (x ) + b f (x ) + 

lira 2 2 n 



. . + b f (x ) - P. 

n n 2 / 2_ 

. . + b f (x ) - P " 

n n ra n 



+ • • • 

— 0 



05 ) 



If each unlaiavm coefficient is to be uniquely detcrr.iincd, the 
nuraber of independent sets of observations rust be equal to or greater 
than the nunber of coefficients. By increasing the nunber of obser- 
vations, a better fit is obtained. 

When raeasureraents are of unequal precision, the least square raetliod 
is e:ctendeu by having the sura of the weighted squai’es of the residuals a 
nininura. 

2 2 2 2 
W j V j = T, l v l + Vf 2 v 2 + * * * + w i< v u = nininun ( 16 ) 

3=1 
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The wciqhts, Wp, Wo, . • . Yf n , depend upon the rccisien iritl i which each 
pair of rccasurencnts is ian.de, tlic mure precise r.casurcr.cntc receiving 
Creator vcichtc. 



6. Tlic Finite Fourier Series as a iSLninun .’’can-Square Appro: imt ion. 

As an introduction to a technique which T.ill be utilised later, 
the least square property of the finite Fburicr series rail be shown. 
Let a function, f(t), which is piecewise continuous end defined fron 
c ^ t ^ c + 2 77 " , be appro:dxiatcd by the function c(t). 



c(t) = ag + ap cos t + bp sin t + ao cos 2t + bg sin 2t 

+ ... + cos nt + b^ sin nt (17) 



n 



c(t) — 'y ^ ^ cos lrt + bj ; sin kt 



( 10 ) 



1 : = 0 



It is desired to choose Oj. and bj, such that the ncan- square error, e(t), 

over the interval for which f(t) is defined is a inininun. 

*c t 277* 



e(t) = 



= ^/0 



f(t) - c(t) 



0 



dt 



mmriiu 



(19) 



For a rininun, the partial derivatives of e(t) with respect to a^ and 
bj. should be aero. The value of c(t) civen by equation (10) is sub- 
st'itutcd into equation (19). Tlic partial derivative with respect to 
my be taken under the intccral sign and th.e rcsultinc egression 

equated to sero. 

4 - 277 * 




n 



-II— R 



f(t) - L (a - cos lrt +■ bj ; sin lrt) 
1: =0 



-E 

L o=o 



cos ot 



dt = 0 (20) 
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a,. cos . t cos jt f b s'n It cog Jt ’t = 

~c 4- 27 7 

f(t) j ccg jt dt (21) 

y c j « 0 

To evaluate the left sit e of e .uati n (Cl), certain orthogonal 

properties of tl.c sine aru cosine ol oild be recalled. 




0 



Lor 



n ^ r.i 



for n = n 



for all n and n 



( 22 ) 

(23) 

(21.) 



The only contribution to the left side of equation (21) occurs when k 

and j are equal. Equation (22) nay no w be written as 

n n re + 2 JT 

Oj, = / £(t) cos let dt (25) 

k*0 1:=0 J c 

As a result of the rearrangement of equation (20) to yet equation (21), 
there is an equality between teres which aid. sc iron substituting the 
sane value of k on each side of equation (25). Hence the general 
corpression for a^ is 

re + ITT 

a^ = -~r f(t) cos Irt dt 



where 1: = 0, 1, . . . n (26) 



This is recognised as the general c:q>ression for the coefficients of 
the cosine terns in the Fourier series e::par.sion. A similar process 
of talzing the partial derivative with respect to will yield the 
general e:<rrc3sion for the coefficients of the sine terms in the Fourier 
series ei-mansion; 



Hi 



The rcraiiinc qucoti.cn to be settled is whether the \~lucc oi' a,, 
and bj. just obtained represent a ...aci. .a, Ida, or s .ddle rout. TIlLs 
is accomplished try finding the values of tie tlrce sec 01 1 part ial t c riv- 
al ives rcpx'cscnlod 1 y A, D, and C at ti.e noJLt (a. , b, ). Ti e di ilex'- 

U 

entiations my ayain be carried out ’inder the irtc-x'al si^n. 




The conditions lor a X'clativc minim of a function of tr.ro variables arc 

B 2 - AC < 0 (30) 

A f C > 0 (31) 

Both conditions are net in this ease. 

Thus if f(t) be piecewise continuous for c ^ t ^ c 4 - 27 7 , then 

the coefficients of the partial sun of the Fourier series of f(t) arc 
precisely tiro sc anony oil coefficients of tire function, q (t), which 
render tire mean- square error a xunimun. 

Tlxc forcq oinq proof was suypcstcd. by a nxroblen in hildebrand-^. 
Both Guillcr.ln^ and ICaplan^ pxrove the same result usinq different 
approaches. 



CHAPTER II All EXTE'.SI 07 LEAST 5s! fAieo IT.IDICTIOII 71 .DORY 



1. Bacic Ac ci" rrtior.c. 

The discussion of this chapter will be based upon th.c general 

problem as stated in Cl-aj ter I. The received, observed, or measured 

signal, R(t), which is the input to the filter, consists of a true 

signal, P(t), and noise, h(t). The time signal, P(t), will be a srunod 

to consist of n completely fordnawn functions of t.Lue, f^(t), fgCt), 

. . . f n (t), each ’multiplied by a constant coefficient, a^, a 0 , . . . 

a„, which is not a function of tir.e. 
n 

P(t) — a^f-^(t) + a^^fa) + » • » ■f a n f n (t) (32) 

n 

p( t) = ^2 a h f h( t ) (33) 

h = l 

ant 

Such functionc as c and si:. a 0 t are not considered since they are in 
effect undetermined functionc. 

From a forelmowlcdgc of the forn of P(t) and ova* operations on 

R(t), we attempt to obtain a closer estimate of P(t), called Q(t), such 

that Q(t) will differ from R(t) by the least amount in core manner or 

other. Tlie ,r best" estimate* Q(t), will be chosen to have the came form 

as P(t), tint is 
n 

Q(t) = bjgj;(t) „ (3U) 

1:=1 

Tlie functions, gj.(t) , in the expression for s»(t) are closely rclatod to 
tlie functions, fj : (t ), in the expression for P(t) and in nary cases may- 
be identical. The ereact restrictions Imposed upon the functions mailing 
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up P(t) and vj( t ) vrlll be disccscd tie : e'~t section. lo assrrr'tion 
Trill be rr.de aboit the type of noise, ’’(t). It iriil *c a ecu. ed tint a 
reasonable criterion o r ‘ ■ crroivnr.ee is one L’i w ich tl-c tL..e weighted 
siuarcd err or is rininiced as illustrated on section three of t. .is 
chapter. Tiis i ini: li nation Is brought a.x;i t bp tic way in which the 
undetermined constants, b^, bo, . . • b n , arc selected. 

In order to si pli!^ the plysical realisation of t..e filters, they 
will be assumed to be constructed of linear element.; i/iiich do net vary 
with tii.ic. li c limited scope of this paper will not permit a t In roup h 
discussion of a broader class of filters vrhicii my be constructed ty 
permitting the cisc of th.c filter clcncnts to vary with time. 

In general, we assume the function, P(t), to be completely fore- 
known. There are exceptions, however, wliich arc a; .enable to treatment 
even if there is some decree of uncertainty as tc the fora, of P(t). 

Tills occurs wiien the function is unknown in the sense that it is one 
of several imown possible functions. The filter nay be designed to tale 
care of all th.c lmown possibilities. As an cm. pile, suppose tliat P(t) 
is hnown to be either ap a2 cS cr a-j e 2 ’*-'. T1 on the actual signal for 
wliiclr tile filter is designed night be 

Q(t) = b]_ + b 2 o' 1 f b^ e 2 ^ (3$) 

Tlic price paid for this uncertainty as to the cm.ct function present is 
an increase in the noise output of the filter. This is so because the 
inclusion of additional functions it1.11 in general require th.c filter to 
have a greater frequency response than that required for the one true 
signal alone. The increase in filter noise output limits the client to 
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vrl.icl. this method can be carried. A: ct or ease is t. at of the an ct^r- 
::±ncd furxtic. n si: (.ft f a 2 ). An it stands, it cannot be treated, 

however, br.- • v.ttinc it in tl.c equivalent fern, ’t>-^ sin irt 4- era wt, 
it my be considered. In this cara ticrc is no dose of c cncr ^i‘ty nor 
increase in the noise output. 



2. Applicable Functions. 

The only assumption which -..'ill be ra.de alxjut the type of functions 
comprising Q(t) is tliat they for.: a closed set under a linear trans- 
lation in tine, that is, no new functions idll be generated. Tills 
restriction results fr^n the fact tint, for simplicity, ti:c filters are 
considered to be invariant v.dth tin. In coir£ fron a static case of 
least squares curve fitting to the dynamic sitration, no tine reference 
is provided the filter. Given a sinusoidal type rave that lias been 
Qoiny on for an indefinitely lone tine, one cannot cay whether the wave 
is a sine or a cosine unless some reference tine ic chosen, however, 
no natter what reference is chosen, the wave can always be described as 

some linear combination of cos wt and sin wt. To understand v.’hat this 

o 

restriction implies, consider the functions sin wt and b2 t under a 
translation in time of duration c. 

b]_ sin w(t f c) = b^ sin wt 4- bj^ cos vrt (36) 

b 2 (t + c) 2 = + b 6 t + b ? t 2 (37) 

In both cases, new functions resulted fron the translations in tir.e. 
how if the c:r. ressions on the riyht sides of equations (3<->) end (37) 
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undergo a further translation in tine, t-ho translations Trill ncrely 

chance the coefficients -without generating ary ncrvr functions. Tims 

the richt sides satisfy the requirement for functions which nal:e up 

Q(t). It nay also be observed that functions of the fora b-^ t^*^ and 

can never satisfy tide restriction. 

b^ t b 2 t t b^ t^ 

The ne:ct question requirinc an answer is, ,f Yfliat functions renain 
a closed set under a linear transfomation in tine?" As an illustration, 
consider the case in which there are two functions involved. 

Q(t) = b^_ Ci(t) t b 2 c 2 (t) ( 38 ) 



The linear transformation nay be written in matrix forn. 



C 2 (t f At) 




A B 




Ci(t) 


g 2 (t f At) 




C D 




c 2 (t) 



But A, B, C, and D nust be chosen so that when At equals zero 



c x (t + 0 ) 




1 0 




C x (t) 


C 2 (t + °) 




0 1 




c 2 (t) 



This condition is satisfied if 



c-j^t + At) 




1 4* a At 


b At 




c x (t) 


g 2 (t 4- At) 




c At 


1 f d At 




c 2 (t) 



( 39 ) 



m 



oa) 



where a, b, c, and d are arbitrary constants. Subtract the natrix 



c x (t) 

c 2 (t) 



from both sides of the natrix equation (Id) and divide try At. 
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(U2) 



g-^(t 4- At) - C]^) 










c x (t) 


At 




a 


b 




C 2 (t + At) - c 2 ( t ) 










C 2 (t) 


At 




c 


a 





Since this oust hold for all At, let At approach the Unit zero. 
The resulting- derivative is denoted by a prine. 



^'(t) 


II 


a b 






6 2 '(t) 




c d 




C 2 (t) 







___ 




_ — 0mmm 



(U3) 



hbvr taking the second derivatives rdth respect to tine. 





II 


a b 






c 2 "(t) 




c d 




e 2 '(t) 


_ 




___ — m 




__ __ 



(UO 



The matrix equations (U3 ) and (Uli) nay novr be used to find sinilar 
second order differential equations in g-^(t) and g ^ ('*')• These equations 
take the form 



8 M (t) + C ' ("t ) + Cg G(t) = 0 vrherc C-^ and C 2 are constants (15) 

By using sinilar Methods, it is evident that if Q(t) consists of n 
functions, these functions nust be solutions of the reduced equation of 
an n th order linear differential equation with constant coefficients. 
This restriction is not as serious as it night first appear since a vide 
variety of useful functions belong in this groug . Included are expo- 
nentials, polynomial s, sinusoids, and functions formed by adding or 
multiplying these types together. Yfnile the functions, f(t), that form 
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the true signal, P(t), my be considered to be ary one solution of the 
differential equation such as sin wt, Q(t) nust consist of the corro- 
soonding complete set of functions, b^ sin wt 4- b 2 cos vrt. Similarly, 
if f(t) lias a polynomial tern such as a 2 t^, g(t) raist contain the corre- 
sponding complete polynomial, t b^ t + b^ t^. 

3. Tine T/eighted Least Squares Curve Fitting. 

The process wirich will be used to obtain a tine weighted least 
squares fit is illustrated in Figure U. The top sketch shows the noisy 
signal, R(t), and the best estimate of tire true signal, Q(t). The 
difference between the two signals is the residual error, R(t) - Q(t). 

In the middle of the figure is ti e square of tl is residual error, 

[R(t) - Q(t)]\ In order to allow some latitude in tire construction of 
tire filter, this squared error is weighted over the past in some arbi- 
trary manner by liultiplying the square of the residual error by a 
weighting function of tire, Ll(t). The rcsilt of the multiplication 
is to produce a tine weighted squared residual error curve as shown 
by the crosslratchcd area of tho bottom diagram. The area underneath 
tliis curve is given by 




It is precisely tlds crosslratchcd area which will be minimised. 

The squared error weighting function, l'(t), ray be thought of as 
the filter’s memory of the squared error. The mathematical restriction 
on U(t) is tint it be chosen so that the area Integral is finite. There 
is, however, another practical restriction on Il(t) . Since one nust wait 
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0 



Figure U. T:L..e Weighted least Squares Curve lotting. 
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■until the snootliing tine, T, liaa been completed or until tlie filter has 
liad tine to settle before it provides an ''optimum” output, the memory 
of the filtor cannot extend too far into the past. M(t) nay be a coo- 
promise between a long snoo tiling tine which makes use of more information 
about the signal and a short tine which would provide a quicker decision 
as to the nature of the signal. 

Probably the three most important forms of M(t) are the rectangular, 
exponential, and damped sinusoidal types shown in Figure 5- In the 
rectangular type, the filter weights all squared errors from a certain 
fined tine in the past, T, up to the present tine equally. This corre- 
sponds to the weighting discussed by Zadeh and Ragazzini-^ in which 

M(t) = u(~t) - u(-t-T) where u(t) is the unit step function (U7) 

The abrupt change in weighting results in filters which contain a delay 
line type element. Another useful type filter is one which forgets what 
has gone on in the past exponentially. Exponential weighting is defined 
ly 

U(t) = e^T u (-t) (U8) 

Tliis type of memory is very promising in that it corresponds to the 
natural da. ping found in most physical systems with dissipative elements, 
and hence holds forth the hope that the filters may be realized with 
lumped constant elements. The third type of weighting defined ty 

M(t) = e^ T (cos wt - sin wt) u(-t) (1*9) 

is an attempt to approximate the rectangular filter. The output of this 
filter would tend to liave a transient damped oscillation and also holds 
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forth the hope of realisation with lumped constant elements. Other 
types of weighting vrhich night find application under certain cir cun- 
stances include t t‘ ; and e^/^ sin wt. 



U. Fornulation of the Tfcighting Function. 

If the expression for Q(t) given in equation ( 3U) is substituted 
into equation (L6), the expression for the crossliatched area in Figure 



U becomes 



= / R(t) - ^ \ Sk (t) 2 

k = l “* 



dt 



m 



The conditions for a minimum arc that the partial derivatives of the 
expression for the area vdth respect to the coefficients b^, b^t • • • 
b n equal zero. This results in a set of n simultaneous linear equations* 



s!£- = o 



* 1 : 



where k — 1, 2, ... n 



( 51 ) 



In order to carry out these differentiations under the integral sign, a 

17 

form of Leibnitz’s rule similar to that given by Kaplan is now stated. 

Let f(x, t) be continuous and have a continuous derivative df/dx in 
a domain of the xt plane which includes the rectangle a x ^ b, 
t^ ^ t ^ tg* Then for a x < b 



~ f f(x, t) dt = f ^ (x, t) dt 
dx / / dx 




( 52 ) 



The partial derivative with respect to the general coefficient, b^, will 
be taken under the integral sign. 
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The change of subscript from k to j vras nac’e to indicate that the 
summations are distinct. Setting the partial derivative equal to zero 



and rearranging the resulting equation 

'O n /'Or n 



Z O n ror n -J ^ 

R(t)^2 c 3 (t) u(t) dt y £b, : c 1: (t) 

* j*l J-oo *-!: = l — ' 3 « = j 

If trro new s/nbols are defined as follows, 

o 



(t) u(t) dt (55) 






R(t) gj(t) U(t) dt 



oo 



Bjk - ®k3 = I Ck<t) e 3 (t) «t) at 

- co 



( 56 ) 

( 57 ) 



then B^can be evaluated since gj,(t), Cj(t)> and M(t) are all known, 
and the set of n simultaneous equations defined by equation (55) take 
the sir.pler form 

3-1 lJ-l j»-j 

In matrix form these equations become 




h 




®U ®12 * * * % n 






h 

• • 


— 


B 21 B 22 * * ' B 2n 




b 2 


In 




B nl ^ 







m 
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Let Cj.-i be the ir. verse matrix of Bjt k , that is 



c n 


c 12 


• * * ^ln 




D 11 


CJ 
1 — i 


* * * 


C 21 


c 22 


• • • c 2n 




Dot 


Boo 

L.C. 


• * * B 2n 


Cnl 


°n2 


• • • c nn 






B n2 


• • • Bnn 



1 0 
0 1 



0 0 . . . 1 



( 60 ) 



Prcnultiplying both sides of equation (59) by the matrix 



Oil C 12 * * * c ln 




1 

M 

H 

1 




H 

1 


C 21 C L2 * * * °2n 




h 


— 


bo 

L. 


C m C n2 * * * C nn 








°n 



(61) 



or, in the form of a summation, 
n 

b k = E ^ 11 (62) 

i«l 

Substituting from equation (62) into equation (3h) 
n n 

Q(t) - E E ii q ci C lc (t) ( 63 ) 

lc-l i=l 

To avoid confusion v/ith the actual variable, t, the dunry variable of 
integration in equation (56) is changed from t to x. The resulting 
value of Ij_ is substituted into ' equation ( 63 ) • 
n n 



Q(t) 



= EE 



c isL ek(t) 



(6U) 



R(x) g i (:;) M(x) dx 

k-1 1=1 L -^~°° 

Rearranging equation (61a) so tliat all terns are under the integral sign, 

/ • O n n 

^ ^ R(-0 Ci(:c) Cj ;i g k (t) I!(x) dx (65) 

-00 1: *=■ 1 i = l 
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Let Cj-i be the i:. verse intri:: of Bjt k > that i; 



C 11 c 12 * * * c ln 
Coi C^o • • • 



°nl °n2 



Cnn 



B*jj^ B-^2 • • • B 



In 



Bo]_ B02 • • • T 



D nl B n2 • • • ?nn 



1 0 . . . 0 

0 1 . . . 0 

0 0 . . . 1 



(CO) 



Prcnultiplyinc both sides of equation ( 59 ) by the matrix 



C 11 


O 

ro 


• * * C ln 


C 21 


C-o 

t . L. 


a 

0 

• 

• 

• 


C nl 


C n2 


• * * C nn 



X 1 

Io 



or, in the form of a summation, 
n 

\ °l:i b. 

i**! 



U 1 

b 0 



n 



(61) 



(62) 



Substituting from equation ( 62 ) ir.to equation ( 3 h) 
n n 

Q(t) — E E Ii Cj ci C k (t) (63) 

lc*l i = l 

To avoid confusion v/ith the actual variable, t, the duixy variable of 
integration in equation ( 56 ) is changed from t to x. The resulting 

value of Ij_ is substituted into' equation (63) • 
n n 



Q(t) = EE E R(x) Gi (:: 

k«l i=l LE-00 



:) M(x) dx 



c lr± Ek^ 



(6U) 



Rearranging equation ( 61 ;) so tliat all terns are under the integral sign, 

/ ■ O n n 

EE B(::) C i (:c) Ci ;i g k (t) M(x) ±z ( 65 ) 

-00 k B l i=l 
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Consider that Q(t) i3 the output of a filter with an input, R(t) . 
Referring to Figure U, it is seen that Q(0) is the best ostimte of 
the true signal at the present time. If t is positive, the value of 
Q(t) is an estimate of a future value of the true signal. Thus the 
quantity t is the prediction tine. For the fixed filters under con- 
sideration, let t be a fixed prediction tine designated by the symbol 
9. Equation (65) nay now be interpreted as a convolution integral of 
the form shown by equation (66). 



Q(e) 



W(6, 0 - x) R(x) dx 



( 66 ) 



-oo 



A comparison of equations (65) and (66) indicates that the weighting 

function of the filter must be 

n n 

W(0, 0 - x) * EE gi (x) Cfei G k (e) M(x) 
k =1 i = l 

Ibldng the substitution 



(67) 



0 - x 



( 68 ) 



the weighting function of the filter becomes 
n n 

v/(e, t) s EE Gj^-t) Cj d g k (0) M(-t) u(t) 

k =1 i= 1 

where u(t) is the unit step function. In matrix fom this becomes 



(69) 



w(e, t) * 

^^(“■b)> £2 ^""^ * * 



C n ( 



*] 



Cil C-^2 

C 21 C 22 



<Txi 



. . C< 



2n 



C nl C n2 • * • C nn 



gl(®) 

e 2 (®) 






M(-t) u(t) 
(70) 
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Since most network synthesis ie done in the frequency domain, it 
is convenient to represent the filter try its complex frequency transfer 
function, Il(s), the Laplace transform of the weighting function. 



CO 



H(s) = / TT(B, t) e"^ dt 



(71) 



To obtain smooth derivatives or integrals, H(s) is either multiplied or 



divided by the variable s. 



5. Steady State Filter Response in the Absence of lloise. 

If the filter weighting functions found in the proceeding section 

are valid, a proper input to the filter in the absence of noise should 

produce a steady state output with no error. Since the noise, ll(t), i3 

zero, the input, R(t), is equal to the true signal, P(t). From equations 

(l) and (33)> the input may be expressed as 
n 

R(t) = Y"' a h f h (t) (72) 

h=l 



Tlie filter weighting function is given by equation (69) as 
n n 

w(e, t) = EE Gi (-t) Cfci g 1: (0) Ll(-t) u(t) 

k-1 i»l 

Fi*om equation (5), the steady state output of the filter is 

r 00 no 

Q(t) = / w(t) R(t - r) dr = / w(-t) R(t * r) dr 

Jo J 00 



(73) 



(7U) 



Substituting the expressions for the input and weighting function into 



the equation for the steady state output, one obtains 
r On n n 

Q(t)-/ ^2 Si(t) C ^ g k (6) U(r) 



U -oo k**l i-1 



y 1 ^ % f h( t 



+ t) 



h=l 



dt (75) 
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Assume tliat 



n 



) fj a (t 4- Z) nay be expressed as 
n * la - n 

E «**" -E E PhtpCO e p (tO 

h— ?. h«— h p =1 

If equation (76) is substituted into equation (75>)» then 
n n n n 

Q(t) ■ IZ CIZ H E k (e) “h v‘f «i w s® 5 u(r) 

k-1 h«l i*l p*l '*■* 



-oo 



From equation (57) the integral B^ p nay be defined as 

B i P “ r° Cp(tO U(t) dr 

J-oo 

This mil reduce equation (77) to the fora 
n n n n 

Qfa) “ ) ) ^ ^ %p ^ki &k(®) % ®hp("0 

k — 1 h=l i=l p=l 
The relation between Bi p and 0^ is 

y~' i Bi p CkL - 1 ^cn p - k 

=* 0 when p ^ k 

Equation (79) nay nor; be reduced to 
n n 

Q(t) - y y ck(®) % Bhk(t) 

k-1 h-1 

By conparison vri. th equation (76) 



n n 



n 




%c(t) e k (8) - Y2, ^ (t * e) 



k-1 h-1 h=l 

so that Q(t) can be expressed as 

n 



Q(t) 



y ^ ah fh(t t 0) 



h-1 



( 76 ) 



d Z 
(77) 



(78) 



(79) 



( 80 ) 



(81) 



( 82 ) 



(83) 
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Thi3 opression nay n ovr be compared vdth equation (72) to give 



Q(t) — R(t + e) (OU) 

Thus the steady state output of the filter equals the filter input 
displaced ty the amount of the prediction tine, b. 

'Die only assumption made in tliia section was that defined by 
equation (76). It leads to a set of equations very similar to the 
matrix set given in equation (39) • Tliis assumption corresponds to 
the requirement that the output of the filter have the sane fom as 
the input except for a possible translation in time. One may again 
conclude tiiat the functions which are amenable to treatment are the 
solutions of the reduced equation of an n th order linear differential 
equation with constant coefficients. 
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CHAPTER III REA.LIZATIOI I OF SUXJTHEJG AIID PREDICTIHG FILTERS 



1. Lfcthods Available for Construction. 

The formulation of the weighting function of Chapter II did not 
insure that every filter was physically realisable using lumped constant 
elcncnts. It did, however, provide such a vn.de variety of squared error 
memory functions, l.I(t), that a high probability exists that the filters 
may bo constructed by one of several methods. 

V/hcn a rectangular error memory function is used, it is often 
convenient to synthesise the transfer functions with operational ampli- 
fiers ouch as those used in analog computers. This is especially true 
if the signal under consideration is composed of lav: frequency com- 
ponents only. Since the rectangular error memory function leads to a 
delay line element, the operational amplifiers have the advantage of 
providing isolation and a means of sunning the undelayed and delayed 
signals. "While conventional delay linen could be used for shorter 
saootliing times, it vrould probably be necessary to resort to a contin- 
uous tapo or drum arrangement such as the one shown in Figure 6 for the 
longer smoothing tines. The actual synthesis of these filters will not 

be discussed in detail since the needed information may be found in 

20 

books on analog computers such as the one 'ey Korn and Korn . 

If an exponential or damped sinusoidal error memory function is 
used, it is often possible to synthesise the filter ao a two terminal 
driving point impedance, Z(s). The input to the filter is a current 
which could be supplied from a pentode while the output is the voltage 
across the terminals. A driving point impedance must be a positive real 
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Erase 




Figure 6. Ibgnetic Tape or Drum Delay Element. 



function. The well lmown conditions for a rational function to be a 
positive real function are 
Z(s) is real when s is real. 

Z(s) is analytic in the right half plane 

Poles on the boundary are simple with positive,’ real residues. 



If the transfer function meets these conditions, the synthesis of the 
filters as driving point impedances may often be done by inspection, 



and Duffin 3 * 10 . 

A second method for synthesizing the transfer function is available 
Then the exponential or damped sinusoidal error memory function is used. 
The transfer function, Il(s), may be considered to be the ratio of the 
open circuit output ■voltage of a two- terminal pair network to the input 
voltage. Kal-oi ' lias shewn the conditions for realising the transfer 
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function to within a constant factor using a finite number of lumped 
constant, linear elencnts. For the symmetrical lattice network these 
conditions are 

H(s) is analytic in the right half s plane. 

Poles on the boundary are simple with residues having a zero real part. 

It is evident that a wider class of functions nay be treated by thi 3 
method than in the case of driving point inpedancos. However, there is 
a practical problem which arises from the fact tliat no common ground 
exists between the input and output terminals. This difficulty Trill not 
be present if the synthesis is carried out as a laddor network. Kahal^ 
lias shown that one additional condition besides those for the symmetrical 
lattice must be satisfied to realise the transfer function to Ydthin a 
constant factor as a ladder network. This condition ±3 that H(s) must 
have no zeros in the right half plane. A major problem in the actual 
synthesis of the filters is usually the factorization of the transfer 
function. Two methods for carrying out the actual synthesis of the 
filter are those of Darlington^, and YTeinbcrg^Tj ‘- (i . 

2. Filter Design Procedure. 

Before proceeding vrith examples of specific filters, it . would be 
well to list the general steps to be followed in the design of ary 
filter. Usually the function(s) to be smoothed, the prediction time, 
and the functional desired are determined by the requirements of the 
problem. 

Step 1. From the known function (s) to be smoothed, find the complete 
set of functions which form a closed set under a translation in tine. 



3U 



If tho smoothed integral of a polynomial is clesirod, the filter must bo 
designed for the next liighcr order polynomial. Conversely, the smoothed 
derivative of a polynomial vri.ll be designed for the next lower order 
polynomial. 

Step 2. Choose a squared error memory function, M(t). 

Step 3* Find the elements of the Bjj, matrix in accordance with equation 
(57). 

Step U. Find the Cj^ matrix vrhich is the inverse of the matrix. 
Ifctliods for obtaining the inverse matrix arc given in the mathematics 
texts by Pipcs^ 4 and Guillcnin^". 

Step 3". Find the numerical values of the functions of step 1 at the 
prediction tine, G. 

•Step 6. Determine the filter weighting function as given by equation 

( 70 ). 

Step 7. Take the Laplace transform of the weighting function. Multiply 
the result by s if a derivative is desired and divide it by s if an 
integral is desired. 

Step 8. Use tho complex frequency transfer function found in step 7 as 
the basis for the synthesis of the network. 

3. Smoothing and Prediction of an Exponential Function. 

The only functions which arc solutions of the first older linear 
differential equation with constant coefficients 

f' f Cp f ~ 0 where Cp is a constant (8£) 

are of the type ap e nl . The steps of the proceeding section will be 
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followed to find filters vdiich both smooth and predict for this function. 

For the first case it is assumed that rectangular weighting of the 
squared error is to be used to smooth the noisy function a]_ The 

best estimate of the function itself is desired with a prediction tine 
of 0 seconds and a smoothing time of T seconds. 



Step 1. g*L = e 13 ^ 



Step 2. 
Step 3. 

Step h. 

Step £. 
Step 6. 

Step 7. 




C 11 = 



2n 



1 - o -2n 



Gx(©) 

w(t) - 

H(s) = 



- 



— c 



2m e 



n(9 - t) 



1 - e 



-2nT 



[u(t) - u(t 



" T)] 



2m e 



m9 



1 - e" 2 ^ 



1 - e^e" 75 
(s + n) 



( 86 ) 

(87) 

( 88 ) 

(89) 

(90) 

(91) 

(92) 



Step 8. One possible method of synthesizing the transfer function is 
shovm in Figure 7. Operational amplifier elements are indicated by 
triangles. Actual values of the elements are not shovm since they depend 
upon 0, T, the type of operational amplifier used and the anticipated 
signal swing. Either the delay element or its associated operational 
amplifier must provide a negative output with respect to the input in 
order tloat the final summing amplifier may perform the subtraction 
indicated in equation (92). 
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Figure 7. Exponential Predictor v/ith Rectangular Vfoighting 



The second case to be illustrated vri.ll be sinilar to the first Tilth 
the exception that exponential v.-eighting vri.ll be used. 



Step 1. 


Si « 


Step 2. 


L f (t) 


Step 3» 


Bn = 


Step it* 


c n = 


Step 5« 


ci(©) 


Step 6. 


w(t) 


Step 7. 


Z(s) : 



ciirt 



2 VT u (_ t ) 



c 2nt e t/T dt 

'-00 

2n + 1/T 



2r. f 1/T 



c.e 



+ n + i/ T 



(93) 

(9U) 

( 95 ) 

(96) 

(97) 

(98) 

(99) 
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Step 6. The synthesis of the filter as a driving point impedance is 
illustrated in Figure 8. 




figure 8. Exponential Predictor vrith Exponential Weighting 

It r.iay be noted tliat when n is negative, the values of T 'which will give 
a realizable network are restricted. Tliis lias physical significance in 
tliat it is necessary for the noise output to decay at a faster rate than 
the signal for a usable output. 

* 

Lu Smoothing of Sinusoidal Functions. 

The functions a^ sin wt, a 2 cos wt, and the linear combination of 
the two are solutions of the second order linear differential equation 
with constant coefficients 

f 1 ’ + C-j_ f' + C 2 f — 0 where C]_ and C 2 ore constants (100) 

In the following e:cample a noisy sinusoidal function with an angular 
frequency of one radian per second is to be smoothed. IIo prediction is 
desired and the error weighting is of the ere -onential type with a 



38 



smoothing time constant of one second 



Step 1. 


Cl = cos t 






(101) 




g2 — sin t 






(102) 


Step 2. 


U(t) = 


e^ u(-t) 






(103) 


Step 3» : 


>]- 


3/5 • 

-V5 


-1/5 

2/5 




(10U) 


Step U. 


M- 


2 1 
1 ’ 3 






(105) 


Step 5. 


si(o) - 


- 1 






(106) 




g 2 (°) = 


- 0 






(107) 


Step 6. 


W(t) = 


(2 cos t 


- 


sin t) e”* 1 u(t) 


(108) 


Step 7. 


Z(s) =* 


2s 1 






(109) 


3 2 f 2s f 2 




Step 8. Figure ? 


shows the 


synthesis of tliis filter 


as a driving point 


impedance. 













As night have been anticipated, the filter is a resonant circuit. 

At an angular frequency of one radian per second its hrtpedance is purely 
resistive and equal to one ohn. The size of the elements should cause 
no alarm since they depend upon the desired frequency and impedance level 
of the filter. For very low frequencies, it might be convenient to use 
the mechanical analogues of the electrical elements. 
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I in 





8/5 henry 



1/2 farad - r 



Ii/3 ohn 



E out 






U/5 ohn 






Figure 9* Filter for Smoothing Sinusoidal Functions 



It appears possible to construct a filter for recurrent square or 
sawtooth waves by designing them for the fundamental frequency and a 
suitable nunber of tlic proper harmonics. A major difficulty with this 
method is that of finding the inverse of a natrio: with a large nunber of 
elements. 

5. Polynomial Filters. 

As an e:au.iplc of the design of polynomial filters, the steps for 
constructing a filter which smooths and predicts a first order poly- 
nomial or ra_p will be shown. The error weighting is of the exponential 
type with a smoothing time constant of or.e second. 



Step 1. 




(no) 




(m) 



Uo 



Step 2 



e t u(-t) 



( 112 ) 



M(t) 



Step 3« 



Step 1;. 



H = 

[di]= 



1 -i 

-1 2 

2 1 

1 1 



step 5. ci(©) = i 



g 2 (e) - e 



step 6. W(t) = [(2 + 9) - t (1 + *)] e _t u(t) 

(2+9) s + 1 



Step 7. Z(s) = 



(s + l ) 2 



(113) 

(nil) 

(n3) 

(U6) 

(n7) 

( 118 ) 



Step 0. The synthesis of the filter as a driving point impedance is 
shown in Figure 10* 




Figure 10. Predictor for a First Order Polynomial 
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Tables 1 and 2 give the inverse matrices for polyno mial s up to and 
including the fifth order. Table 1 is for rectangular weighting with a 
smoothing time of 1 second while Table 2 is for exponential weighting 
with a snootiiing time constant of 1 second. If T is other than 1 second, 
the tables nay still be used by scaling the problen properly. For 
example, the expression for the weighting function of a predictor for a 
first order polynomial using rectangular weighting and a smoothing tine 
of 1 second, is 



vr(t) 



[■ 



lif 6e - (6 + 12 e) t 



u(t) 



- u(t - 1)J 



(119) 



and the corresponding expression for a general smoothing tine T is 



W(t, T) = - 
T 



b + 6 - - (6+12-)i 



u(t) - u (t - T)j 



( 120 ) 



Figures 11 and 12 show the family of polynomial smoothing filters 
derived from Table 2. These filters are of the low pass type with an 
increasing response to higher frequencies as the order of the polynomial 
is increased. The complex frequency transfer functions of the n th 
order polynomial smoothing filters shown in Figures 11 and 12 have been 
found to take the form 

ju -f 1 

H(s) = 1 - (121) 

(s fl) n+1 



Figure 13 shows the amplitude and phase response versus frequency 
for first order polynomial predictors similar to the one in Figure 10. 
Figure lb shows the sane information about the polynomial filters in 
Figures 11 and 12. A positive increase in prediction tine or polynomial 
order evidently tends to increase the noise output of the filter. 
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Table 1 



Inverse or M Matrices for Polynonials of Order m. 
Ll(t) = u(-t) - u( — t—l) 



n ** 0 


[0 








9 


36 


30 


n = 2 


36 


192 


180 




30 


180 


180 



a = 1 



U 6 

6 12 



16 


120 


2li0 


1U0 


120 


1200 


2700 


1680 


2U0 


2700 


6U80 


1-200 


lho 


1680 


U200 


2800 



25 


300 


1050 


lliOO 


630 






300 


U800 


18900 


26880 


12600 






10^0 


18900 


79380 


117600 


56700 






Uoo 


26880 


117600 


179200 


88200 






630 


12600 


56700 


88200 


Uooo 






36 


630 


3360 


7560 


7560 


2112 


630 


1U700 


88200 


211680 


220500 


83160 


3360 


88200 


5610-80 


11Q1200 


1512000 


582120 


7560 


211680 


1101200 


3628800 


3969000 


1552320 


7560 


220500 


1512000 


3969000 


hlOOOOO ' 


17b6360 


2772 


8316O 


582120 


1552320 


17U636O 


6985bli 
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Table 1. 



Inverse or Matrices for Polynonials of Order n. 

LI(t) = u(-t) - u(--Ul) 



n — O £ 1 J 



9 


36 


30 


36 


192 


180 


30 


180 


180 



ra =■ 1 



U 6 

6 12 



16 


120 


2h0 


lliO 


120 


1200 


2700 


1680 


2U0 


2700 


6U80 


1*200 


mo 


1680 


U200 


2800 



25 


300 


1050 


moo 


630 






300 


U800 


18900 


26880 


12600 






1050 


18900 


79380 


117600 


56700 






moo 


26880 


117600 


179200 


88200 






630 


12600 


56700 


88200 


UlOO 






36 


630 


3360 


7560 


7560 


2772 


630 


1U700 


08 200 


211680 


220500 


83160 


3360 


88200 


56Ui80 


1)11200 


1512000 


582120 


7560 


211680 


1)11200 


3628800 


3969000 


1552320 


7560 


220500 


1512000 


3969000 


ItilOOOO 


17U6360 


2772 


83160 


582120 


1552320 


17U6360 


6905m 
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Table 2 



Inverse or 




Ll(t) — u(-t) 



Matrices for Polynomials of Order m. 



n = 0 




m — 1 



2 

1 



1 

1 



n = 2 



3 3 1/2 

3 5 1 

1/2 1 1/U 



U 6 2 

6 lU 11/2 

2 11/2 5/2 

1/6 1/2 1/U 



5 


10 


5 


5/6 


1 / 2 U 


10 


30 


35/2 


19/6 


V6 


5 


35/2 


23/2 


9A 


1/8 


5/6 


19/6 


9/U 


17/36 


1/36 


1/2U 


1/6 


1/8 


1/36 


1/576 



6 


15 


10 


15/6 


15 


55 


85/2 


23/2 


10 


85/2 


73/2 


127/12 


15/6 


23/2 


127/12 


13A 


1/U 


29/2U 


7/6 


3/8 


1/120 


1/21- 


1/21- 


1/72 



vu 

29/2U 

7/6 

3/8 

13/2C8 



1/6 

1/2 

1/U 

1/36 



1/120 

1/2U 

1/2U 

1/72 

1/576 



1/576 1/lUUOO 





n = 2 m — 3 

Figure 11. Zero Through Third Order Polynomial Smoothing Filters. 

M(t) ~ e^ u(-t) 

All resistances, inductances, and capacitances are given in ohns, henries, 
and farads respectively. ' 
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Figure 12. Fourth and Fifth Order Polynorial Smoothing Filters. 



16 



Amplitude Response **syonse 171 De 8 rees 




'' r -v.: c 7". .V' 1': L 1 ■ t .77'. r _ 

!' 




Amplitude Response lfca.se ifespcsnse In repress 





6. Filtors for a Combination of Functions. 

Since ary additive or multiplicative combination of sinusoids* 
exponentials, and polynomials is a solution to a reduced equation of 
some linear differential equation with constant coefficients, it nay 

4 * 4 - 

be handled. Such functions as t sin t, e sin t, and t e sin t may 
thus be treated after the complete set of functions forming a closed 
set upon translation in time is found. A filter which is the additive 
combination of a first order polynomial and a sinusoid can be designed 
as follows. 



Step 1. 


gl = cos 


t 










(122) 




g 2 — sin t 










(123) 




g 3 =1 












(121;) 




§U =t 












(125) 


Step 2. 


H(t) = e 


u(-t) 










(126) 


Step 3. 




3/5 


-V5 


1/2 




0 






r 1 


-V5 


s/5 


-1/2 


1/2 


















(127) 




L J J 


1/2 


-1/2 


1 




-l 








0 


V 2 


-1 




2 




Step It. 




12 


-It 


-lit . 


•6 








r “1 


-U 


8 


8 


2 








Ki = 












(128) 




L kl J 


-Hi 


8 


20 


8 










-6 


2 


8 


Li 







h9 



Step 5 



gx(0) = 1 



(129) 





g 2 (o) =o 


(130) 




g 3 (o) =1 


(131) 




g U (o) =0 


(132) 


Step 6. 


W(t) = J^6 - 2t - 2 cos t - U sin tj e"*' u(t) 


(133) 


Step 7* 


__ Us3 + 6s <i -f 6s + 2 

-f ljs-^ + 7s^ + 6s + 2 


(13b.) 


Step 8. 


Figure 15 illustrates one method of realizing this filter 


as 



a driving point impedance. 




Figure 15« Smoothing Filter for a Ranp and Sine Wave. 
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Fhon equation (13U) 



Z(jl) = 1 (135) 

z(0) « l (136) 

Thus in the steady state the driving point impedance for a sinusoidal 
input vdth an angular frequency of one radian per second or for a 
constant input is a pure resistance of one olm. Yfhen the input is 
equal to t, the output nay be found by multiplying Z(s) by the Laplace 

p 

transform of t, that is l/ s -» The inverse Laplace transform of the 
product lias a steady state tern equal to t. The steady state output 
divided by the input is then t/t or 1. Although the filter is con- 
structed vdth reactive elements, its steady state impedance to a 
sinusoid, a constant, and a ramp input appears to be a pure resistance 
of one ohm* A phenomenon akin to resonance is thus observed for 
functions of tine other than sine raves. 



7* Response of Various Polynomial Kilters in the Absence of lloise. 

In order to evaluate the polynomial filters, the response of the 
filters to standard input signals in the absence of noise can be 
calculated from equation (5) • The standard signals chosen for this 
evaluation are 



% = 1 



(a constant) 



(137) 



R 2 = 1 + t (a ramp) (138) 

R 3 = 1 4- t t t 2 



(a quadratic) 



(139) 



The graphs on the following pages show the desired response to these 
signals as dashed lines and the actual response as solid linos* Jbr 
smoothing filters the desired steady state responses to Rp and R^ 
are 1, 1ft, and If t f t^; for integrating filters, t, t f t^/2, 
and t + t^/2 f t3/3i and fbr differentiators, 0, 1, and 1 + 2t respec- 
tively. 

Figure 3 16 through 21 inclusive illustrate the response of smoothing 

I 

filters designed for constants, ramps, and quadratics with both rectan- 
gular and exponential weighting. It should be noted that the steady- 
state response of the rectangular filter is reached abruptly at the end 

I 

of the smoothing period while the exponential filter requires five to 
ten time constants. Filters designed to smooth an ra th order polynomial 
■will also smooth lower order polynomials. If a polynomial of order 
if qf 1 is put into the filter, the output will have an error which is 
a polynomial in t of order q, where q is a positive constant. 

The response of polynomial smoothing filters to a unit step input is 
illustrated in Figures 22 and 23* As the order of the polynomial for 
which the filter is designed increases, the overshoot and frequency of 
transient oscillation is observed to increase. 

Figures 2h, 25, and 26 show the response of filters designed to 
predict a ramp function with four different values of prediction time 
and three different error memory functions. The influence of the error 
memory function upon tlie transient response can be clearly seen. 

Responses of filters derived by dividing the complex frequency trans- 
fer functions, H(s), for the constant and ramp smoothing filters by the 
variable s are shown in Figures 27 through 30* These curves indicate that 
a filter which is to integrate polynomials up to the m th order should be 



based upon polynomial snootliing filters of the n f p th order, where p is 
the limber of integrations to be performed. This night be anticipated 
fron the fact tliat an integrator increase. 1 ; the order of t!:c outnut poly- 
nomial as conpared with the input polynomial* 

Differentiators liave responses as illustrated by Figures 31 through 
36. These filters differentiate all polynomials up to the order m if they 
are based on polynomial snootliing filters of at least the order n - q, 
where q is the number of differentiations. If the rectangular or crqion- 
ential error memory functions are used, the res’ or.se curves show discon- 

v 

tinuities in the output. These discontinuities indicate that the filters 
have an infinite Frequency response and hence would not be very practical 
for eliminating high frequency noise components. The discontinuities 
result fron the fact tiiat the derivative of a unit step function is the 
unit impulse or Dirac delta function and may be eliminated by choosing 
error memory functions which never liave an infinite slope. The responses 
of two filters having such error memory functions are shown in Figures 35 
and 36 . 
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